Here we provide a descriptive overview of the syllable identifications relative to target (table 1). In the current data the number of syllables identified by EasyAlign perfectly matched the targeted number of syllables, i.e., in 100% of the trials there were 0 differences in the number of syllable detected versus target. Note however that we will manually check the EasyAllign syllable detections for the final dataset.
Table 1. Percentage of syllable detection mismatches| syllable differences | percentage |
|---|---|
| 0 | 100.00 |
Table 2 provides the percentages of different type of L2 stress placement matches and mismatches.
Table 2. Percentage of correct L2 placements| stress mis/match type | stress difference | percentage |
|---|---|---|
| L2 correct | same | 33.93 |
| L2 incorrect & L1 match | same | 0.00 |
| L2 incorrect & L1 mismatch | same | 16.07 |
| L2 correct | difference | 44.05 |
| L2 incorrect & L1 match | difference | 0.00 |
| L2 incorrect & L1 mismatch | difference | 5.95 |
For the first analysis we simply assess whether the absolute difference in stress timing relative to the target stress time is different for the gesture or the no gesture condition.
D$accuracy <- abs(D$stressed_mistimingL2L1) #absolute deviation from stress from L2
#basemodel predicting the overall mean accuracy
model0 <- lme(accuracy~1, data = D, random = list(~1|ppn, ~1|target), method = "ML", na.action = na.exclude)
#alternative model with gesture versus no gesture as predictor
model1 <- lme(accuracy~condition, data = D, random = list(~1|ppn, ~1|target), method = "ML", na.action = na.exclude)
anova(model0, model1) #test difference basemodel versus model 1## Model df AIC BIC logLik Test L.Ratio p-value
## model0 1 4 4069.241 4084.509 -2030.620
## model1 2 5 4066.132 4085.217 -2028.066 1 vs 2 5.109432 0.0238summary(model1) ## Linear mixed-effects model fit by maximum likelihood
## Data: D
## AIC BIC logLik
## 4066.132 4085.217 -2028.066
##
## Random effects:
## Formula: ~1 | ppn
## (Intercept)
## StdDev: 0.011073
##
## Formula: ~1 | target %in% ppn
## (Intercept) Residual
## StdDev: 58.9668 85.65074
##
## Fixed effects: accuracy ~ condition
## Value Std.Error DF t-value p-value
## (Intercept) 40.91071 8.046698 167 5.084162 0.0000
## conditionnogesture 21.28571 9.373199 167 2.270912 0.0244
## Correlation:
## (Intr)
## conditionnogesture -0.582
##
## Standardized Within-Group Residuals:
## Min Q1 Med Q3 Max
## -1.7144678 -0.4332536 -0.1847360 -0.1847359 4.1253449
##
## Number of Observations: 336
## Number of Groups:
## ppn target %in% ppn
## 2 168It is of course possible that there is a contextual effect of gesture, depending on whether there is a stress difference or a presence of an accent (see Figure 2)
Figure 2.
We will further assess this in a complex model we expand our analysis with the relevant stimuli conditions, as well as their interactions with the gesture condition. If the interactions are statistically reliable we will perform a post-hoc comparisons with R-package “lsmeans” with a bonferroni correction.
#alternative model with gesture versus no gesture as predictor
model2 <- lme(accuracy~condition+stress+accent, data = D, random = list(~1|ppn, ~1|target), method = "ML", na.action = na.exclude)
model3 <- lme(accuracy~condition+stress+accent+
condition*stress+
condition*accent,
data = D, random = list(~1|ppn, ~1|target), method = "ML", na.action = na.exclude)
anova(model1, model2, model3) #test difference basemodel versus model 1## Model df AIC BIC logLik Test L.Ratio p-value
## model1 1 5 4066.132 4085.217 -2028.066
## model2 2 7 4067.404 4094.124 -2026.702 1 vs 2 2.727168 0.2557
## model3 3 9 4070.265 4104.619 -2026.132 2 vs 3 1.139482 0.5657#summary model 3 post hoc
sum3 <- summary(model3)
posthoc3a <- lsmeans(model3, list(pairwise ~ condition|stress), adjust="bonferroni")
posthoc3b <- lsmeans(model3, list(pairwise ~ condition|accent), adjust="bonferroni")```r
sum3
```
```
## Linear mixed-effects model fit by maximum likelihood
## Data: D
## AIC BIC logLik
## 4070.265 4104.619 -2026.132
##
## Random effects:
## Formula: ~1 | ppn
## (Intercept)
## StdDev: 0.009251582
##
## Formula: ~1 | target %in% ppn
## (Intercept) Residual
## StdDev: 58.19645 85.36076
##
## Fixed effects: accuracy ~ condition + stress + accent + condition * stress + condition * accent
## Value Std.Error DF t-value
## (Intercept) 52.35119 13.93053 165 3.758017
## conditionnogesture 9.94048 16.27766 165 0.610682
## stressdifference -21.29762 16.08559 164 -1.324018
## accentaccent present -1.58333 16.08559 164 -0.098432
## conditionnogesture:stressdifference 3.00000 18.79582 165 0.159610
## conditionnogesture:accentaccent present 19.69048 18.79582 165 1.047599
## p-value
## (Intercept) 0.0002
## conditionnogesture 0.5423
## stressdifference 0.1873
## accentaccent present 0.9217
## conditionnogesture:stressdifference 0.8734
## conditionnogesture:accentaccent present 0.2964
## Correlation:
## (Intr) cndtnn strssd accntp cndtn:
## conditionnogesture -0.584
## stressdifference -0.577 0.337
## accentaccent present -0.577 0.337 0.000
## conditionnogesture:stressdifference 0.337 -0.577 -0.584 0.000
## conditionnogesture:accentaccent present 0.337 -0.577 0.000 -0.584 0.000
##
## Standardized Within-Group Residuals:
## Min Q1 Med Q3 Max
## -1.60812794 -0.46910650 -0.23332169 -0.08683539 4.28520679
##
## Number of Observations: 336
## Number of Groups:
## ppn target %in% ppn
## 2 168
```
```r
posthoc3a
```
```
## $`lsmeans of condition | stress`
## stress = same:
## condition lsmean SE df lower.CL upper.CL
## gesture 51.6 11.4 1 -93.0 196
## nogesture 71.3 11.4 1 -73.2 216
##
## stress = difference:
## condition lsmean SE df lower.CL upper.CL
## gesture 30.3 11.4 1 -114.3 175
## nogesture 53.0 11.4 1 -91.5 198
##
## Results are averaged over the levels of: accent
## Degrees-of-freedom method: containment
## Confidence level used: 0.95
##
## $`pairwise differences of condition | stress`
## stress = same:
## 2 estimate SE df t.ratio p.value
## gesture - nogesture -19.8 13.3 165 -1.489 0.1385
##
## stress = difference:
## 2 estimate SE df t.ratio p.value
## gesture - nogesture -22.8 13.3 165 -1.714 0.0883
##
## Results are averaged over the levels of: accent
## Degrees-of-freedom method: containment
```
```r
posthoc3b
```
```
## $`lsmeans of condition | accent`
## accent = no accent:
## condition lsmean SE df lower.CL upper.CL
## gesture 41.7 11.4 1 -102.8 186
## nogesture 53.1 11.4 1 -91.4 198
##
## accent = accent present:
## condition lsmean SE df lower.CL upper.CL
## gesture 40.1 11.4 1 -104.4 185
## nogesture 71.2 11.4 1 -73.3 216
##
## Results are averaged over the levels of: stress
## Degrees-of-freedom method: containment
## Confidence level used: 0.95
##
## $`pairwise differences of condition | accent`
## accent = no accent:
## 2 estimate SE df t.ratio p.value
## gesture - nogesture -11.4 13.3 165 -0.861 0.3906
##
## accent = accent present:
## 2 estimate SE df t.ratio p.value
## gesture - nogesture -31.1 13.3 165 -2.342 0.0204
##
## Results are averaged over the levels of: stress
## Degrees-of-freedom method: containment
```In the previous analyses we know whether speech prosody performance increases or decreases as a function of gesture, stress difference, and accentedness. A further question is whether gesture-speech synchrony is affected by stress difference and accentedness.
Figure 3.
It seems that synchrony between gesture and speech is best in the accented and stress match condition.
```
## Model df AIC BIC logLik Test L.Ratio p-value
## model0 1 4 2044.734 2057.230 -1018.367
## model1 2 6 2047.489 2066.233 -1017.745 1 vs 2 1.2451018 0.5366
## model2 3 7 2049.444 2071.312 -1017.722 2 vs 3 0.0448689 0.8322
```Now we should know whether gesture-speech synchrony can be affected by trial condition that may complicate correct stress placement. If indeed gesture-speech synchrony is affected, we can wonder about how gesture and speech diverge when they are more asynchronous. Firstly assess whether gestures
#basemodel predicting the overall mean accuracy
model0 <- lme(asynchrony_L2L1~1, data = subD, random = list(~1|ppn, ~1|target), method = "ML", na.action = na.exclude)
#alternative model with gesture versus no gesture as predictor
model1 <- lme(asynchrony_L2L1~stress*correct, data = subD, random = list(~1|ppn, ~1|target), method = "ML", na.action = na.exclude)
anova(model0, model1) #test difference basemodel versus model 1## Model df AIC BIC logLik Test L.Ratio p-value
## model0 1 4 2133.622 2146.118 -1062.811
## model1 2 7 2126.648 2148.515 -1056.324 1 vs 2 12.97427 0.0047summary(model1) ## Linear mixed-effects model fit by maximum likelihood
## Data: subD
## AIC BIC logLik
## 2126.648 2148.515 -1056.324
##
## Random effects:
## Formula: ~1 | ppn
## (Intercept)
## StdDev: 0.007961735
##
## Formula: ~1 | target %in% ppn
## (Intercept) Residual
## StdDev: 130.1349 2.115984
##
## Fixed effects: asynchrony_L2L1 ~ stress * correct
## Value Std.Error DF
## (Intercept) 14.14516 16.72969 163
## stressdifference 30.74958 22.54347 163
## correctL2 incorrect & L1 mismatch 49.44575 32.69010 163
## stressdifference:correctL2 incorrect & L1 mismatch -210.84048 58.87324 163
## t-value p-value
## (Intercept) 0.845512 0.3991
## stressdifference 1.364013 0.1744
## correctL2 incorrect & L1 mismatch 1.512560 0.1323
## stressdifference:correctL2 incorrect & L1 mismatch -3.581262 0.0005
## Correlation:
## (Intr) strssd cLi&Lm
## stressdifference -0.742
## correctL2 incorrect & L1 mismatch -0.512 0.380
## stressdifference:correctL2 incorrect & L1 mismatch 0.284 -0.383 -0.555
##
## Standardized Within-Group Residuals:
## Min Q1 Med Q3 Max
## -0.063643564 -0.007597103 -0.002391494 0.008405780 0.050266435
##
## Number of Observations: 168
## Number of Groups:
## ppn target %in% ppn
## 2 168https://link.springer.com/article/10.3758/s13428-021-01546-0